Expert system for prediction of changes to local environment

ABSTRACT

Disclosed is a system, method, and computer program product that employs high dynamic range (HDR) image processing and manipulation algorithms for capturing and measuring real-time sky conditions for processing into control input signals to a building&#39;s automated fenestration (AF) system, daylight harvesting (DH) system and HVAC system. The photometer comprises a color camera and a fitted fish-eye lens to capture 360-degree, hemispherical, low dynamic range (LDR) color images of the sky. Both camera and lens are housed in a sealed enclosure protecting them from environmental elements and conditions. In some embodiments the camera and processes are controlled and implemented by a back-end computer.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 13/798,050 (now U.S. Pat. No. 9,406,028) that claims benefit ofU.S. Application No. 61/696,052 filed 31 Aug. 2012, the contents ofwhich is hereby expressly incorporated in its entirety for all purposes.

BACKGROUND OF THE INVENTION

The present invention relates generally to a building automation controlsystem, and more specifically, but not exclusively, to use of a highdynamic range (HDR) sky map for predictive control.

This disclosure relates to a photometric device controlled wirelessly ordirectly by a microcontroller and/or a back-end computer system tocontrol a building's automated daylighting fenestration system.

The subject matter discussed in the background section should not beassumed to be prior art merely as a result of its mention in thebackground section. Similarly, a problem mentioned in the backgroundsection or associated with the subject matter of the background sectionshould not be assumed to have been previously recognized in the priorart. The subject matter in the background section merely representsdifferent approaches, which in and of themselves may also be inventions.

It has been determined that the use of daylight harvesting (DH) toreplace or supplement electric lighting in buildings can result insignificant energy and demand savings as well as improve comfort andvisual performance. For example, DH may be accomplished using lightingcontrol systems that are able to dim or switch electric lighting inresponse to changing daylight availability. High performancefenestration systems are a necessary element of any successfuldaylighting design that aims to reduce lighting energy use. Newfenestration technologies have been developed that aim at controllingthe intensity of the incoming solar radiation, its interior distributionand its spectral composition, as well as thermal losses and gains. Forbest performance these fenestration systems often incorporate automatedcomponents such as, but not limited to, shades, Venetian blinds,interior/exterior fixed and adjustable louvers, electrochromic glazings,and optical components (i.e., light redirecting devices) in order torespond to the dynamic nature of daylight and its component parts ofdirect sun, diffuse sky and exterior objects reflecting on to thefenestration. These controls are with respect to openings or portals ina building or wall envelope, such as for windows, doors, louvers, vents,wall panels, skylights, storefronts, curtain walls, and slope glazedsystems.

Automated fenestration (AF) systems use a combination of photometers,pyranometers, and computer algorithms to measure and predict real timesun and sky conditions in order to control how these systems modulatenatural daylight illumination in the interior spaces of buildings whilepreventing glare, heat-gain and brightness discomfort for the building'soccupants. Current fenestration control systems, like SolarTrac byMechoSystems, employ an array of exterior mounted pyranometers andphotometers to measure sky conditions and sky brightness' from abuilding's roof top as well as from specific façade orientations (e.g.north, east, south, west). The measured irradiance values from the roofare compared against published theoretical values for specific latitudesto determine if the sky condition is clear or overcast. When the sky isovercast the shades are raised. When clear the system adjusts the shadesof each fenestration orientation according to the solar geometry forthat orientation and the desired depth of direct sun allowed to enter into the space. The photometric values of sky brightness, measuredvertically from discrete façade orientations, are compared againstspecified luminance levels to determine if shades need to be closed forcontrol of visual and thermal comfort.

Unfortunately these systems require multiple pyranometers andphotometers, each capable of taking only very specific measurements(i.e. irradiation or illuminance) of only the global component of thesky (i.e. the diffuse sky and solar contribution are measured together).Without the ability to sample the sky directionally and discreetlyclouds cannot be discerned from the clear sky component to determinesuch metrics as amount of cloud coverage, cloud size and brokenness ofcloud coverage and the amount of direct, solar diffusion caused by thecloud. These measurements are necessary for approximating and predictingif, when, and for how long the sun is or may be occluded by clouds.Without the latter capabilities, AF systems tend to either not react intime, or to overreact when control is or is not needed.

Additionally, the façade mounted photometers inability to separatelymeasure the direct solar component and diffuse sky component atdifferent facade orientations impedes their ability to control for glareand direct solar gain. The effects of these two components on visualglare and thermal gain are different, requiring each to be measuredseparately. Sun hitting a photometer at an angle to its receivingsurface's orientation will cause a high photometric reading, but may notbe a source of glare if the angle is such that the circumsolar region isout of the visual field of the occupants. In contrast, a relativelylower photometric reading of a bright, overcast sky in the visual fieldof building occupants may be high enough to be a source of visualdiscomfort.

Furthermore, the integration of photometers and pyranometers withfenestration systems is costly and complicated limiting its market shareand the benefits associated with it.

What is needed is a system and method for measurement and accurateprediction of sky/weather influence on building systems that respond tolocal sky-related environment changes.

BRIEF SUMMARY OF THE INVENTION

Disclosed is a system and method for a system and method for measurementand accurate prediction of sky/weather influence on building systemsthat respond to local sky-related environment changes.

The following summary of the invention is provided to facilitate anunderstanding of some of the technical features related to predictivebuilding automation control systems, and is not intended to be a fulldescription of the present invention. A full appreciation of the variousaspects of the invention can be gained by taking the entirespecification, claims, drawings, and abstract as a whole. The presentinvention is applicable to other sky-influenced predictive controlsystems, as well as other environments such as indoor and outdoorlighting systems.

An apparatus including an HDR capturer for obtaining an image of a localenvironment; a mapper for processing said image to extract a pluralityof metrics and performance data of said local environment; and an expertlearning system, responsive to said plurality of metrics and performancedata to generate a near real-time prediction of a local change in saidlocal environment and initiating a change in alocal-environment-influencing system to counter said local change.

A computer-implemented method, including a) producing an HDR image for alocal environment; b) extracting, using a computing system, a pluralityof metrics and performance data of said local environment from said HDRimage; and c) predicting, using said computing system, a local change insaid local environment responsive to said plurality of metrics andperformance data of said local environment from said HDR image.

An embodiment of the present invention may include multiple HDRcapturers for obtaining images of a local environment. A mapper mayprocess these images to extract a plurality of metrics and performancedata of said local environment; and an expert learning system,responsive to said plurality of metrics and performance data to generatea near real-time prediction of a local change in said local environmentand initiating a change in a local-environment-influencing system tocounter said local change.

An apparatus including an HDR (high dynamic range) capturer configuredto obtain an image of a local environment, the HDR capturer including atleast a pair of horizontally arranged HDR imagers each having afield-of-view wherein the field-of-view of each the HDR imager isnon-aligned with at least one other field-of-view of the at least thepair of imagers; a mapper processing the image and extracting aplurality of metrics and performance data of the local environment; andan expert learning system, responsive to the plurality of metrics andperformance data generating a near real-time prediction of a localchange in the local environment and initiating a change in alocal-environment-influencing system to counter the local change.

A computer-implemented method, including a) producing an HDR (highdynamic range) image for a local environment, the HDR image producedfrom an HDR capturer including at least a pair of horizontally arrangedHDR imagers each having a field-of-view wherein the field-of-view ofeach the HDR imager is non-aligned with at least one other field-of-viewof the at least the pair of imagers; b) extracting, using a computingsystem, a plurality of metrics and performance data of the localenvironment from the HDR image; and c) predicting, using the computingsystem, a predicted local change in the local environment responsive tothe plurality of metrics and performance data of the local environmentfrom the HDR image, wherein the predicted local change is in advance ofan actual local change in the local environment wherein the steps a)-c)are included in a control system for a building, wherein the buildingincludes an automated climate control responsive to the control system,and wherein the predicted local change is processed into control signalsfor the control system which operates the automated climate control toreduce a magnitude of the actual local change or a rate of change of theactual local change.

An apparatus including an HDR (high dynamic range) capturer configuredto obtain an image of a local environment, the HDR capturer including atleast a pair of horizontally arranged HDR imagers each having afield-of-view wherein the field-of-view of each the HDR imager isnon-aligned with at least one other field-of-view of the at least thepair of imagers; a mapper processing the image and extracting aplurality of metrics and performance data of the local environment; andan expert learning system, responsive to the plurality of metrics andperformance data generating a near real-time assessment of the localenvironment and initiating a change in a local-environment-influencingsystem to respond to the assessment.

A computer-implemented method, including a) producing an HDR (highdynamic range) image for a local environment, the HDR image producedfrom an HDR capturer including at least a pair of horizontally arrangedHDR imagers each having a field-of-view wherein the field-of-view ofeach the HDR imager is non-aligned with at least one other field-of-viewof the at least the pair of imagers; b) extracting, using a computingsystem, a plurality of metrics and performance data of the localenvironment from the HDR image; and c) assessing, using the computingsystem, the local environment producing an assessment and initiating achange in a local-environment-influencing system to respond to theassessment.

A computer-implemented method, including a) producing an HDR (highdynamic range) image for a local environment; b) extracting, using acomputing system, a plurality of metrics and performance data of thelocal environment from the HDR image; and c) assessing, using thecomputing system, the local environment producing an assessment andinitiating a change in a local-environment-influencing system to respondto the assessment.

A non-transitory computer readable medium with computer executableinstructions stored thereon executed by a processor to perform themethod of assessing a local environment, the method comprising: a)producing an HDR (high dynamic range) image for a local environment, theHDR image produced from an HDR capturer including at least a pair ofhorizontally arranged HDR imagers each having a field-of-view whereinthe field-of-view of each the HDR imager is non-aligned with at leastone other field-of-view of the at least the pair of imagers; b)extracting, using a computing system, a plurality of metrics andperformance data of the local environment from the HDR image; and c)assessing, using the computing system, the local environment producingan assessment and initiating a change in a local-environment-influencingsystem to respond to the assessment.

As described herein, many embodiments include a predictive element. Thatis, the systems and methods make a near-real-time assessment of thelocal environment for controlling a building system, with those controlmechanisms tuned for predicting what the local environment will be likesome time in the future such as to allow for some of the buildingautomation systems to make advance preparations. Some of the systemsrequire some advance time to efficiently make whatever adaptations andcorrections will preferably maintain the building within desiredoperating parameters. This is a special case of the more generalizedprocess of responding to the local assessment of the local environmentas some systems may require little if any lead time. In these cases, anembodiment of the present invention may appear to be more “reactive”than “predictive” but as described herein, the predictions may beconsidered a near-real-time reaction to the current local environment.

Any of the embodiments described herein may be used alone or togetherwith one another in any combination. Inventions encompassed within thisspecification may also include embodiments that are only partiallymentioned or alluded to or are not mentioned or alluded to at all inthis brief summary or in the abstract. Although various embodiments ofthe invention may have been motivated by various deficiencies with theprior art, which may be discussed or alluded to in one or more places inthe specification, the embodiments of the invention do not necessarilyaddress any of these deficiencies. In other words, different embodimentsof the invention may address different deficiencies that may bediscussed in the specification. Some embodiments may only partiallyaddress some deficiencies or just one deficiency that may be discussedin the specification, and some embodiments may not address any of thesedeficiencies.

Other features, benefits, and advantages of the present invention willbe apparent upon a review of the present disclosure, including thespecification, drawings, and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying figures, in which like reference numerals refer toidentical or functionally-similar elements throughout the separate viewsand which are incorporated in and form a part of the specification,further illustrate the present invention and, together with the detaileddescription of the invention, serve to explain the principles of thepresent invention.

FIG. 1 illustrates an embodiment of a HDR sky mapping system;

FIG. 2 illustrates a system architecture for an automated control systemincluding environment prediction;

FIG. 3 illustrates a generic environment-informed predictive controlsystem;

FIG. 4 illustrates a flowchart of a generic environment-informedpredictive control process;

FIG. 5 illustrates an alternative system architecture for determiningcloud speed and direction; and

FIG. 6 illustrates an imaging system including two HDR imagers.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention provide a system and method for asystem and method for measurement and accurate prediction of sky/weatherinfluence on building systems that respond to local sky-relatedenvironment changes. The following description is presented to enableone of ordinary skill in the art to make and use the invention and isprovided in the context of a patent application and its requirements.

Various modifications to the preferred embodiment and the genericprinciples and features described herein will be readily apparent tothose skilled in the art. Thus, the present invention is not intended tobe limited to the embodiment shown but is to be accorded the widestscope consistent with the principles and features described herein.

Definitions

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this general inventive conceptbelongs. It will be further understood that terms, such as those definedin commonly used dictionaries, should be interpreted as having a meaningthat is consistent with their meaning in the context of the relevant artand the present disclosure, and will not be interpreted in an idealizedor overly formal sense unless expressly so defined herein.

The following definitions apply to some of the aspects described withrespect to some embodiments of the invention. These definitions maylikewise be expanded upon herein.

As used herein, the term “or” includes “and/or” and the term “and/or”includes any and all combinations of one or more of the associatedlisted items. Expressions such as “at least one of,” when preceding alist of elements, modify the entire list of elements and do not modifythe individual elements of the list.

As used herein, the singular terms “a,” “an,” and “the” include pluralreferents unless the context clearly dictates otherwise. Thus, forexample, reference to an object can include multiple objects unless thecontext clearly dictates otherwise.

Also, as used in the description herein and throughout the claims thatfollow, the meaning of “in” includes “in” and “on” unless the contextclearly dictates otherwise. It will be understood that when an elementis referred to as being “on” another element, it can be directly on theother element or intervening elements may be present therebetween. Incontrast, when an element is referred to as being “directly on” anotherelement, there are no intervening elements present.

As used herein, the term “set” refers to a collection of one or moreobjects. Thus, for example, a set of objects can include a single objector multiple objects. Objects of a set also can be referred to as membersof the set. Objects of a set can be the same or different. In someinstances, objects of a set can share one or more common properties.

As used herein, the term “adjacent” refers to being near or adjoining.Adjacent objects can be spaced apart from one another or can be inactual or direct contact with one another. In some instances, adjacentobjects can be coupled to one another or can be formed integrally withone another.

As used herein, the terms “connect,” “connected,” and “connecting” referto a direct attachment or link. Connected objects have no or nosubstantial intermediary object or set of objects, as the contextindicates.

As used herein, the terms “couple,” “coupled,” and “coupling” refer toan operational connection or linking. Coupled objects can be directlyconnected to one another or can be indirectly connected to one another,such as via an intermediary set of objects.

The use of the term “about” applies to all numeric values, whether ornot explicitly indicated. This term generally refers to a range ofnumbers that one of ordinary skill in the art would consider as areasonable amount of deviation to the recited numeric values (i.e.,having the equivalent function or result). For example, this term can beconstrued as including a deviation of ±10 percent of the given numericvalue provided such a deviation does not alter the end function orresult of the value. Therefore, a value of about 1% can be construed tobe a range from 0.9% to 1.1%.

As used herein, the terms “substantially” and “substantial” refer to aconsiderable degree or extent. When used in conjunction with an event orcircumstance, the terms can refer to instances in which the event orcircumstance occurs precisely as well as instances in which the event orcircumstance occurs to a close approximation, such as accounting fortypical tolerance levels or variability of the embodiments describedherein.

As used herein, the terms “optional” and “optionally” mean that thesubsequently described event or circumstance may or may not occur andthat the description includes instances where the event or circumstanceoccurs and instances in which it does not.

As used herein, the term “size” refers to a characteristic dimension ofan object. Thus, for example, a size of an object that is spherical canrefer to a diameter of the object. In the case of an object that isnon-spherical, a size of the non-spherical object can refer to adiameter of a corresponding spherical object, where the correspondingspherical object exhibits or has a particular set of derivable ormeasurable properties that are substantially the same as those of thenon-spherical object. Thus, for example, a size of a non-sphericalobject can refer to a diameter of a corresponding spherical object thatexhibits light scattering or other properties that are substantially thesame as those of the non-spherical object. Alternatively, or inconjunction, a size of a non-spherical object can refer to an average ofvarious orthogonal dimensions of the object. Thus, for example, a sizeof an object that is a spheroidal can refer to an average of a majoraxis and a minor axis of the object. When referring to a set of objectsas having a particular size, it is contemplated that the objects canhave a distribution of sizes around the particular size. Thus, as usedherein, a size of a set of objects can refer to a typical size of adistribution of sizes, such as an average size, a median size, or a peaksize.

One embodiment of the HDR photometer system 110 is illustrated in FIG. 1(section view). As depicted therein, the broad aspects of the embodiment110 includes at least one interchangeable camera 112 capable ofcapturing a sequence of LDR color images (or frames) 214 at differentshutter speeds and/or apertures and having a minimum of one circularfish-eye lens 114 capable of providing a 360-degree azimuthal view and180-degree or more hemispherical view of the sky. The camera 112 andlens 114 are housed in an environmentally protected enclosure 116 so asto permit the camera 112 to capture a 360-degree azimuthal view and180-degree or more hemispherical view of the sky 128.

The camera 112 is operatively connected 122 to a back-end computer 124that instructs the camera 112, via software, to capture a sequence ofLDR images 214 of the sky 128 at different shutter speeds at a presetinterval of time. Typically a sequence of five, exposure and or aperture(F-stop) bracketed images are captured at a rate of one sequence per twominutes but one may reduce this or increase this number of brackets andrate at which they are taken depending on user data needs. The color,exposure bracketed images are transmitted to the computer 124 forprocessing into a single HDR radiance map 220 of the sky.

The preferred camera 112 used in this invention is a digital cameracapable of being controlled and interfaced 122 physically with acomputer 124 (e.g. RS-232, USB, IEEE 1394, TCP/IP and GIGE) orwirelessly (e.g. Bluetooth, Wi-Fi, ZigBee and wireless USB or others).However other embodiments may use other camera types such as, but notlimited to, digital or analog video cameras where the video signal isoutputted via an analog or digital signal to a computer with anappropriate interface board. The signal will then be input into a framegrabber board mounted on a computer (not shown).

In still another embodiment of the present invention a weatherproofcamera 112 and lens 114 is employed without the need for anenvironmentally protected enclosure 116.

In another embodiment of the present invention the camera 112 is cooledand heated directly to maintain the camera's 112 temperature within themanufacturer's recommended operating temperature ranges by attachedthermoelectric (e.g., Peltier cooler/heater) modules 118 controlled by atemperature sensor. Other means of cooling and heating the camera mayinclude, but are not limited to, fans, heat sinks, liquid cooling pumps,and small electric heaters.

In still another embodiment of the present invention the camera 112 orlens 114 are fitted with a filter 120 (e.g. Neutral Density filter,Polarizing filter, Near Infrared filter and colored filters). Thesefilters may be used to protect the camera's image senor from thepossible harmful effects of the sun and or to enhance specific aspectsof the images produced.

In still another embodiment of the present invention the camera 112 isoperatively connected 122 to a microprocessor 126 enclosed in the sameenvironmentally protected enclosure 116. The microprocessor 126 takesthe place of the back-end computer 124 mentioned previously.

In still another embodiment of the present invention a pyranometerand/or photometer (not shown) are employed. These sensors are attachedto or are in close proximity to the environmentally protected enclosure116 to capture global values of sun and sky irradiation and orilluminance respectively.

FIG. 2 conceptually illustrates the software architecture 210 employedto control the system 110 as well as the procedures and transformationsemployed for calculating cloud, sun and sky photometric data that'slater processed into control signals sent to the automated fenestration(AF) 130, daylight harvesting (DH) 132 and heating, ventilation, and airconditioning (HVAC) 134 systems. Once triggered by a user-set timeinterval the camera control software (CCS) 212 instructs the camera 112to capture a single LDR image 214 at the camera's 112 fastest exposuresetting. The captured image is analyzed to determine if the averagepixel brightness meets a predetermined, minimum value appropriate forinclusion in the processing of the final HDR radiance map 220. If theimage is determined to be too dark then the camera is instructed tocapture another image at a lower exposure until a captured image meetsthe minimum, average pixel brightness. Once an acceptable image iscaptured more images are acquired, each at sequentially longer exposuretimes. This embodiment captures 5 LDR images 214, however more or lessthan this can be captured to meet a user's needs. Also, in someembodiments of the present invention both exposure and or aperturebrackets, rather than exposure alone, may be used when capturing the LDRsequence of images. Different CCS 212 may be used for this embodimentincluding an Astro IIDC program for Apple Computer's operating systemOSX by Aupperle Services and Contracting, or FlyCapture SDK program forLinux operating systems by Point Grey Research Inc. However, othercamera control programs are available for OSX, Linux, and Windows.

After a sequence of exposure bracketed, LDR images 214 are acquired fromthe system 110 by computer 124, metadata information EXIF 216 of eachLDR image 214 is updated, if necessary, with the exposure speed, cameraISO speed and the lens f-stop setting. Next, an HDR image generator 218processes the LDR images and EXIF 216 into an HDR radiance map 220(e.g., a “sky map”). The preferred HDR image processing software used inHDR image generator 218 includes “HDRgen” written by Greg Ward ofExponent Corporation. The pixel data in the HDR radiance map containreal-world luminance values equivalent to those that might be measuredusing a spot luminance meter. Once processed the HDR radiance map 220 isrun through several algorithmic and ray-tracing procedures (e.g.,procedure 222-procedure 228) to extract quantitative cloud, sky andsolar data. These procedures may include: a radiance map sampling andprocessing procedure 222, a sky patch and zone generation procedure 224,a cloud filtering and masking procedure 226, and a cloud edge-to-arearatio procedure 228. Procedures 230-232 may be calculated independentlyfrom HDR radiance map 220 for determining solar position and global anddirect values of horizontal and vertical solar irradiation/illuminancevalues for the buildings location at the same date and time as theacquisition of LDR images 214. Procedures 230-procedure 232 may include:a solar position procedure 230 and a solar irradiance and illuminanceprocedure 232.

In procedure 222 the HDR radiance map's 220 pixel values are processedand sampled for calculating the diffuse, horizontal illumination (lux orlm/m²) value at the photometer as well as the highest recorded pixelvalue captured in the solar region of the HDR radiance map. The highestrecorded pixel value is divided by the highest achievable senor pixelvalue, for the particular camera used, in order to provide a reductionand correction factor for use in later procedures such as thecalculation of the amount by which the sun's direct beam component isdiffused by cloud cover and for global illumination/irradiationpredictions.

In procedure 224 the HDR radiance map 220 is subdivided into discreetpatches similar to the method described by Tregenza P R. 1987.Subdivision of the sky hemisphere for luminance measurements. LightingResearch and Technology. Vol 19:13-14. These patches are thenreassembled to represent specific zones of the sky. The number of zonesgenerated is chosen by the user based on the number of building façadeorientations. Each of these reassembled zones contains all the pixelbrightness values for within that portion of the sky visible to abuilding occupant looking through a building's fenestration towards thatspecific building orientation. Pixels not within these specific viewboundaries are given a value of zero. Finally, an average sky brightness(cd/m²) is calculated for each of these zone (pixels values of zero areignored).

In procedure 226 the HDR radiance map 220 is filtered and masked toisolate clouds from sky using a fixed threshold algorithm fordetermining fractional sky cover in a way similar to that presented byLong, C. N., J. M. Sabburg, J. Calbo, and D. Pages, 2006: Retrievingcloud characteristics from ground-based daytime color all-sky images. J.Atmos. Oceanic Technol, 23, 633-652. Specifically pixels with ared-to-blue signal ratio (R/B) greater than a predetermined, fixed valueare classified as cloud, while lower values of the R/B are classified ascloud-free. Once all cloud/clear pixels have been determined, thefractional sky cover is calculated as the number of cloud pixels dividedby the total number of pixels in the HDR radiance map 220 (any bordermasks surrounding the fish-eye, HDR radiance map 220 are ignored).

In procedure 228 the HDR radiance map 220 is filtered to isolate pixelson the cloud/clear-sky boundaries to determine the cloud edge-to-arearatio. Specifically, the number of pixels on the cloud/clear-skyboundaries are divided by the total number of pixels within all clouds.This value indicates average cloud size and brokenness of cloudcoverage. A high edge-to-area ratio is indicative of broken clouds ofsmall diameter, while a smaller ratio results from extended clouds.

In procedure 230 the solar profile angle is calculated for the location,time and date of the HDR radiance map 220 for all building façadeorientations. The solar profile angle is derived from the altitude andazimuth angles of the sun's position.

In procedure 232 theoretical, clear sky, solar irradiance andilluminance values are calculated for the building's location at thesame time and date as the HDR radiance map 220 on both the horizontalplane and the vertical planes of the building's façades orientations.The algorithms used in these calculations are similar but not specificto the methods described by Perez, R., Ineichen, p. and Seals, R. (1990)Modeling daylight availability and irradiance components from direct andglobal irradiance. Solar Energy 44, 271-89, hereby expresslyincorporated by reference. These theoretical values are then adjusted byfactoring them with the calculated value for solar region'smeasured-pixel/brightest-pixel-achievable ratio from procedure 222. Theresultant values are approximations of real-time, solar illuminance andirradiance relative to the current sky conditions. Adding the solar,horizontal illuminance value to the measured value of diffuse horizontalilluminance from procedure 222 gives global horizontal illuminance. Aphotometer, either attached to or are in close proximity to theenvironmentally protected enclosure 116, can also be used for directlymeasuring global horizontal illuminance.

In procedure 234 calculated data from the above procedures are processedinto control signals such as may be sent to the building's AF 130, DH132 and HVAC 134 systems, for example. For the AF 130 system, skyinformation is quantified (and saved for future calculations) fordetermining whether clouds are occluding the solar region and, if so, bywhat amount the direct sun is diffused. When occluded and the HDR map'ssolar region measured-pixel/brightest-pixel-achievable ratio fromprocedure 222 is below a predetermined amount, then the valuescalculated for percent global sky overcast 226, the amount of cloudbrokenness 228, the sun's path 230 and the percent cloud content in thedirection of movement (determined through comparison with previouslysaved results) and the sun's path are called. These results areprocessed and compared against previously saved results for determininghow long and, in the case of many broken clouds, how frequently the sunwill be occluded. When the calculated period of time or frequency atwhich the sun will be occluded is above a user-defined threshold, thensky brightness at all fenestration orientations are calculated andcompared against a predetermined threshold level for visual glare. Basedon these calculated cloud conditions and measured sky brightnessreadings the AF 130 controls are signaled to respond (e.g. shadingsystems that would otherwise be drawn or closed to control for directsun and/or thermal gain are opened at windows not oriented towards abright sky glare condition). Where electrochromic glass or other phasechange glass is used in the AF 130 system, the cloud predictionalgorithm signals the AF 130 controls to change the glass's phase orstate of tint (to account for the inherent time lag period associatedwith these changes) in anticipation of forecasted clearing skies anddirect sun.

When the sun is determined to be un-occluded by clouds, the verticalsolar irradiance (W/m²) on and sky brightness (cd/m²) from allfenestration orientations are calculated and compared againstpredetermined threshold levels for solar heat gain and visual glare. Inaddition, current and previously saved results for cloud coverage.Location, brokenness, speed and direction are analyzed for determiningif current cloud conditions may occlude the sun within a predeterminedperiod of time relative to the solar path of the sun and the percentcloud content in a direction-speed vector. Based on these measured andcalculated results the AF 130 controls are signaled to respond byadjusting the fenestration to predetermined glare control presets or tothe profile angle of the sun relative to the fenestration's orientationand user-set depth of direct sun desired to enter the space. Whereelectrochromic glass is used in the AF 130 system, the cloud predictionalgorithm signals the AF 130 controls to change the glass's state oftint (to account for the inherent time-lag associated with these changesin tint level) in anticipation of the advancing clouds and the resultantoccluding of the solar region.

For the DH 132 system, measured and calculated data from the aboveprocedures signals the DH 132 system when the measured and calculateddaylight values reach a predetermined level to signal a change inelectric lighting switching or dimming. The DH 132 system is initiallycalibrated on-site using handheld photometers for each of the building'sfloor levels.

For the HVAC 134, calculated values from procedure 232 for vertical,global irradiation on the building's facades orientations are used todetermine solar heat gain entering the building. These calculated valuesinform the HVAC system of current thermal loads as well as projectedloads based on the cloud prediction algorithm in procedure 234.

For meteorological system 136 measured and calculated data will be madeavailable in a larger context (multiple devices over a largergeographical area) over the internet or other communications system thatwill be able to provide a high resolution of sky related phenomena(e.g., cloud cover, solar radiation, and the like) to allow anunderstanding of that data in real time to allow microclimateprediction. The combination of this data plus readily availablemeteorological data will allow expert systems to be able to time amountsof radiation, precipitation, cloud movement, and wind conditions at amicroclimatic level. A microclimate is a local atmospheric zone wherethe climate differs from the surrounding area. The term may refer tovery small areas, for example a garden bed, or as large as many squaremiles. Microclimates exist, for example, near bodies of water which maycool the local atmosphere, or in heavily urban areas where brick,concrete, and asphalt absorb the sun's energy, heat up, and reradiatethat heat to the ambient air: the resulting urban heat island is a kindof microclimate.

For utility system 138, architecture 210 provides advance information ofbuilding performance and energy requirements for a predetermined time inthe future. For example, a prediction that in the very near future anoon-time sun will emerge from behind heavy cloud cover (or that thenoon-time sun will become occluded by heavy cloud cover), and that thiscondition may persist for a particular amount of time. Based upon otheroccupancy, use, and modeling information associated with the building,this advance data allows the utility to quantitatively, in nearreal-time, understand and respond to energy demand increases/decreases.The scope of this understanding is not limited to the building and itsmicro-climate, but may be representative of other nearby building energyrequirements and expected changes. Of course architecture 210 mayinclude a plurality of systems 110 may be distributed strategicallyacross many buildings and allow the utility to have an even largeraggregate near real-time map of upcoming aggregated energy demand withenough time that the utility may respond appropriately (in increasingoutput to avoid energy shortage or decreasing output to save costs)among other possible response modalities.

FIG. 3 illustrates a generic environment-informed predictive controlsystem 300, of which system 110 is a particular example. System 300includes an HDR image acquisition subsystem (e.g., HDR capture) 305 thatobtains, either through operation of one or more imaging systems asdescribed herein, or use of other suitable data, an HDR image of thesky/local environment. Suitable data varies based upon implementation,but preferably includes near real-time image acquisition sufficient forthe desired prediction/forecast window of sky/local environment events.

System 300 also includes a sky mapper 310 that processes the HDR imageto extract metrics and characterizations of the sky/local environment.These metrics include elements important to the prediction/forecastingsuch as, for example, procedures 222-232 shown in FIG. 2. In someimplementations, sky mapper 310 may have additional or fewer elementsthan described herein.

System 300 includes an expert system 315 (also referred to as a learningsystem) that uses the metrics and characterizations provided by skymapper 310 in one or more models and predictive systems. These modelsand predictive systems may be simple or quite complex depending upon theparticular use. In some implementations for example, expert system 315includes a model of a thermal performance of a building in response tovarious environment loads, operational and lead time requirements forbuilding automation systems 320 (e.g., automated fenestration, daylightharvesting, and HVAC control(s)) or information collection/production325. In addition, a horizon-to-horizon path of the sun, local buildingsand their influence/input into important variables, and other specificinformation of the building and its operation in the local environmentthat are important to expert system 315 are used as necessary ordesirable. Most preferably expert system 315 is implemented as alearning system to develop and improve prediction and forecasting as itstudies how the building and its subsystems react to various parametersit measures and/or calculates.

Control system(s) 320 often benefit from advance input of up-comingsky/local environment events because of lead-time to achieve a desiredresponse. For example, an electrochromic window may take 15 minutes todarken sufficiently in response to a command change. Expert system 315provides control system 320 for the electrochromic window with advanceinformation enabling it to be in a proper mode in response to aparticular event. Expert system 315 informs control system 320 to darkenat least 15 minutes before the sun conditions change in any way thatwould inform its darkness characteristic. In another instance amechanical cooling system may take some time to alter the thermalsensation of an interior space. Advance warning may allow the system togently cool the space as it heats up as opposed to trying to remove theheat once the space is warm enough to trigger a thermostat.

Information system(s) 325 aggregate and/or transmit measured/calculateddata. Stored information may be used to improve expert system 315 orcharacterize building or a local micro-climate that includes thebuilding. Transmission of measured/calculated data, particularly in nearreal-time enables superior energy and lighting management, particularlyof the building and other appropriate areas that react similarly to thetransmitted data.

FIG. 4 illustrates a flowchart of a generic environment-informedpredictive control process 400. Process 400 is preferably implemented bysystem 110 or 310 or other similar system. Process 400 begins with astep 405 of producing an HDR local environment image (e.g., a series ofsets of sky images for a building). After step 405, process 400 extractsenvironment metrics and characterizes performance of the environment andelements in the environment at step 410. For example, determine wherethe sun is in the sky, whether there is behind cloud cover, how thecloud cover affects the sunlight with respect to the building, and whenthe sun will emerge from behind the cloud cover. Process 400 nextpredicts control requirements and/or environment performance at step415. While these are often related, they are not always the same assometimes process works in cooperation with a local control system andother times the predictions/forecasts are used in other ways, some ofthose ways may be a remote control system.

FIG. 5 illustrates an alternative system architecture 500 fordetermining cloud speed and direction. Architecture 500 is modified fromarchitecture 210 illustrated in FIG. 2 to measure cloud speed anddirection (i.e., cloud velocity). Architecture 500 is substantially thesame as architecture 210 in arrangement and operation except whereexpressly identified or where it is clear from context. Architecture 500includes camera control software 505 that is modified from cameracontrol software 212 of architecture 210 for determining cloud velocity.Architecture 500 also includes a set of applications 510 thatencompasses controllable building automation systems and other systems510. Systems 500 include the systems and applications illustrated aselements 130-element 138, and may include other systems and applicationsin addition.

As further described herein, architecture 500 includes an additional setof elements 515 for determining cloud velocity that is controlled bycamera control software 505. After software 505 uses the imager tocapture a first set (e.g., five) of LDR images 214, software 505operates the imager to capture a second set (e.g., three) of LDR images520. There may be many different ways to determine cloud velocity fromthe imager and FIG. 5 illustrates one representative methodology.

After a sequence of exposure bracketed, LDR images 214 are acquired fromthe system 110 by computer 124, second set of LDR images 520 areacquired by camera 112 at a user set interval using an automaticexposure set by the CCS 505. This and other embodiments similar to theone illustrated capture 3 LDR images 520 at two-second intervals usingan automatic exposure setting, however more or fewer images can becaptured using shorter or faster intervals to meet a user's needs. LDRimages 520 are transformed from a hemispherical projection to a planarprojection 525 which are then passed through an optical flow analysisroutine 530 for calculating cloud speed and direction 535. Using thecalculated values of cloud speed and direction 535 a vector ofpredetermined pixel width is drawn from the known sun position 230 inthe last of the LDR image sequences 520 and extended out in timesegments in the direction from which the clouds are moving and at adistance based on speed given by 535. This embodiment uses 3,five-minute vector segments of speed to define a total vector distanceof fifteen minutes, but longer or shorter time intervals can be chosenby the user. Finally, the percentage of cloud to clear sky pixels withineach five-minute vector segment is calculated 540 and used for cloud andsolar irradiation forecasting 15 minutes out. This percentage vectorovercast of 540 is provided to data processing 234 for use in thesystems and applications generally identified in systems 510.

FIG. 6 illustrates an imaging system including multiple horizontal HDRimagers (e.g., 2). As illustrated in FIG. 1, and as employed in thearchitectures of FIG. 2 and FIG. 5, a single vertical HDR imager may beemployed to satisfactory result. In some circumstances, performance maybe improved by including multiple (e.g., 2, 3, 4, 5, 6, 7, 8, 9, 10, ormore) HDR imagers, with at least two being horizontally disposed. Asingle vertical HDR imager almost exclusively processes the sky vault.In some building automation systems (BAS), significant radiation andilluminance contributions may come from indirect paths other thandirectly from the sun. Having multiple horizontal imagers thatcollectively define a spherical field-of-view from a point of interest(e.g., a top of a building) encompasses not only the sky vault but alsothe ground vault which includes direct and indirect sources that mayaffect the BAS of a building. Additionally, some rooftops includevarious obstructions (e.g., a spire or the like) that may degradeperformance of some single vertical HDR imager solutions.

A number and an orientation of each HDR imager of system 600 may beinfluenced by many factors. In some situations, e.g., away from theequator, orienting a pair of imagers into north-south fields-of-view maybe advantageous. For installations closer to the equator, there may bean advantage to orienting the pair of imagers with east-westfields-of-view.

Many buildings include four exterior faces that are generally orientednorth, east, south, and west. The arrangements of the imagers may beadjusted so that they are positioned at a top of the building at theperimeter with the field-of-view of each individual imager normal to aplane of a face of the building. A typical four-face building may havetwo or four individual imagers at the top of the building along itsperimeter and facing horizontally outward normal to two (or three, orfour) faces. A “triangular” building may include three HDR imagers, a“Pentagon” may include five individual imagers, and so on for differentnumbers of faces of buildings.

In system 210 of FIG. 2, camera control software 212 is modified toactuate each individual imager (preferably concurrently with otherimagers of any particular imaging solution but time-staggered may besufficient in some cases) to capture the desired number of LDR images214. For a dual-imager solution, system 210 initially captures 10individual LDR images 214 which includes 5 pairs of LDR images.Similarly, a triple-imager solution produces 15 individual LDR images214 which includes 5 triplets of LDR imagers. System 210 merges thepairs (or the triplets) or the individual concurrent LDR image from eachindividual imager's contribution from the multiple imagers of the HDRimaging solution. At each exposure time, system 210 creates one, singleLDR image providing a 360 degree horizontal and vertical, spherical,radiance map of the complete, local environment. The HDR radiance map isthen constructed from these single LDR images as described herein.

With respect to system 500 illustrated in FIG. 5, not only may there bemultiple LDR 214 images at each exposure time, there may be multiple LDR520 images at each exposure time. In each case when there are multipleimages from one exposure time, system 500 may combine the multiple LDRimages to provide a single 360 degree horizontal and vertical,spherical, radiance map of the complete, local environment. Processingthen may proceed using the enhanced LDR images.

System 600 may include multiple, interchangeable cameras 612 and lenses614 (not to scale in FIG. 6) that are employed and positionedhorizontally to face specific building façade orientations 616. Eachcamera within this system 610 captures an orientation specific,360-degree azimuthal view and 180-degree or more hemispherical view, HDRimage of the sky 628 as well as the local site 620 and ground conditions622 above and below the horizon 624. The acquired HDR images can becombined into one, single image providing a 360 degree horizontal andvertical, spherical, radiance map of the complete, local environment.

The system and methods above has been described in general terms as anaid to understanding details of preferred embodiments of the presentinvention. In the description herein, numerous specific details areprovided, such as examples of components and/or methods, to provide athorough understanding of embodiments of the present invention. Somefeatures and benefits of the present invention are realized in suchmodes and are not required in every case. One skilled in the relevantart will recognize, however, that an embodiment of the invention can bepracticed without one or more of the specific details, or with otherapparatus, systems, assemblies, methods, components, materials, parts,and/or the like. In other instances, well-known structures, materials,or operations are not specifically shown or described in detail to avoidobscuring aspects of embodiments of the present invention.

Reference throughout this specification to “one embodiment”, “anembodiment”, or “a specific embodiment” means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment of the present invention and notnecessarily in all embodiments. Thus, respective appearances of thephrases “in one embodiment”, “in an embodiment”, or “in a specificembodiment” in various places throughout this specification are notnecessarily referring to the same embodiment. Furthermore, theparticular features, structures, or characteristics of any specificembodiment of the present invention may be combined in any suitablemanner with one or more other embodiments. It is to be understood thatother variations and modifications of the embodiments of the presentinvention described and illustrated herein are possible in light of theteachings herein and are to be considered as part of the spirit andscope of the present invention.

It will also be appreciated that one or more of the elements depicted inthe drawings/figures can also be implemented in a more separated orintegrated manner, or even removed or rendered as inoperable in certaincases, as is useful in accordance with a particular application.

Additionally, any signal arrows in the drawings/Figures should beconsidered only as exemplary, and not limiting, unless otherwisespecifically noted. Furthermore, the term “or” as used herein isgenerally intended to mean “and/or” unless otherwise indicated.Combinations of components or steps will also be considered as beingnoted, where terminology is foreseen as rendering the ability toseparate or combine is unclear.

As used in the description herein and throughout the claims that follow,“a”, “an”, and “the” includes plural references unless the contextclearly dictates otherwise. Also, as used in the description herein andthroughout the claims that follow, the meaning of “in” includes “in” and“on” unless the context clearly dictates otherwise.

The foregoing description of illustrated embodiments of the presentinvention, including what is described in the Abstract, is not intendedto be exhaustive or to limit the invention to the precise formsdisclosed herein. While specific embodiments of, and examples for, theinvention are described herein for illustrative purposes only, variousequivalent modifications are possible within the spirit and scope of thepresent invention, as those skilled in the relevant art will recognizeand appreciate. As indicated, these modifications may be made to thepresent invention in light of the foregoing description of illustratedembodiments of the present invention and are to be included within thespirit and scope of the present invention.

Thus, while the present invention has been described herein withreference to particular embodiments thereof, a latitude of modification,various changes and substitutions are intended in the foregoingdisclosures, and it will be appreciated that in some instances somefeatures of embodiments of the invention will be employed without acorresponding use of other features without departing from the scope andspirit of the invention as set forth. Therefore, many modifications maybe made to adapt a particular situation or material to the essentialscope and spirit of the present invention. It is intended that theinvention not be limited to the particular terms used in followingclaims and/or to the particular embodiment disclosed as the best modecontemplated for carrying out this invention, but that the inventionwill include any and all embodiments and equivalents falling within thescope of the appended claims. Thus, the scope of the invention is to bedetermined solely by the appended claims.

What is claimed as new and desired to be protected by Letters Patent ofthe United States is:
 1. An apparatus comprising: an HDR (high dynamicrange) capturer configured to obtain an image of a local environment,said HDR capturer including at least a pair of horizontally arranged HDRimagers each having a field-of-view wherein said field-of-view of eachsaid HDR imager is non-aligned with at least one other field-of-view ofsaid at least said pair of imagers; a mapper processing said image andextracting a plurality of metrics and performance data of said localenvironment; and an expert learning system, responsive to said pluralityof metrics and performance data generating a near real-time assessmentof said local environment and initiating a change in alocal-environment-influencing system to respond to said assessment. 2.The apparatus of claim 1 wherein said near real-time assessment includesa prediction of a local change in said local environment and whereinsaid initiating a change in said local-environment-influencing systemincludes initiating a counter to said local change.
 3. The apparatus ofclaim 2 wherein said local environment includes a sky view from abuilding exterior, said sky view including a near real-timecharacterization of a sun, a sun location, and a path of said sun insaid sky view, and a position, a velocity, an acceleration, and anopacity of a sun-occluding object that influences said near real-timeprediction by said expert system.
 4. The apparatus of claim 3 whereinsaid sun-occluding object includes a cloud disposed in said sky viewalong said path of said sun.
 5. The apparatus of claim 3 wherein saidHDR capturer includes an HDR photometer.
 6. The apparatus of claim 5wherein said HDR photometer includes a camera configured to capture aplurality of sequences of low dynamic range (LDR) color images, eachimage of a particular sequence having one of a different aperture or adifferent shutter speed as compared to other images of said particularsequence, said camera capturing said sequence of images through a fieldof view accessed by a fish-eye lens of said camera, said field-of-viewconfigured to provide a 360-degree azimuthal view and a 180-degree ormore horizon-to-horizon view of said sky view.
 7. The apparatus of claim6 wherein each said sequence includes five LDR color images and eachsaid sequence is captured approximately every two minutes.
 8. Acomputer-implemented method, comprising: a) producing an HDR (highdynamic range) image for a local environment, said HDR image producedfrom an HDR capturer including at least a pair of horizontally arrangedHDR imagers each having a field-of-view wherein said field-of-view ofeach said HDR imager is non-aligned with at least one otherfield-of-view of said at least said pair of imagers; b) extracting,using a computing system, a plurality of metrics and performance data ofsaid local environment from said HDR image; and c) assessing, using saidcomputing system, said local environment producing an assessment andinitiating a change in a local-environment-influencing system to respondto said assessment.
 9. The computer-implemented method of claim 8wherein said assessing includes predicting a local change in said localenvironment producing a predicted local change and wherein saidinitiating said change in said local-environment-influencing systemincludes initiating a counter to said local change.
 10. Thecomputer-implemented method of claim 9 wherein said steps a)-c) areincluded in a control system for a building, wherein said buildingincludes an automated climate control responsive to said control system,and wherein said predicted local change is processed into controlsignals for said control system which operates said automated climatecontrol to reduce a magnitude of said actual local change or a rate ofchange of said actual local change.
 11. The computer-implemented methodof claim 10 wherein said automated climate control includes an automatedfenestration system for one or more portals of said building.
 12. Thecomputer-implemented method of claim 10 wherein said automated climatecontrol includes a heating, ventilation, and air conditioning (HVAC)system for one or more occupancy zones within said building.
 13. Thecomputer-implemented method of claim 10 wherein said automated climatecontrol includes a daylight harvesting system for one or more occupancyzones within said building.
 14. The computer-implemented method of claim9 wherein said steps a)-c) are included in a distributed meteorologicalsystem having a plurality of data gathering locations distributed over ageographical region having a plurality of microclimates greater innumber than a number of said data gathering locations, wherein each saiddata gathering location performs steps a)-c), and wherein a statisticalprocessing of said predicted local changes for one or more of saidmicroclimates.
 15. The computer-implemented method of claim 9 whereinsaid steps a)-c) produce a predicted local change for a first buildingconnected to a power grid operated by a utility company, wherein saidpredicted local change for said first building is accompanied by achange in an energy demand by said first building through said powergrid wherein said change in said energy demand determines whether saidutility changes an energy production for said power grid, and whereinsaid predicted local change is provided to said utility allowing saidutility to modify said energy production consistent with said actuallocal change.
 16. A computer-implemented method, comprising: a)producing an HDR (high dynamic range) image for a local environment; b)extracting, using a computing system, a plurality of metrics andperformance data of said local environment from said HDR image; and c)assessing, using said computing system, said local environment producingan assessment and initiating a change in a local-environment-influencingsystem to respond to said assessment.
 17. The computer-implementedmethod of claim 16 wherein said assessing includes predicting a localchange in said local environment producing a predicted local change andwherein said initiating said change in saidlocal-environment-influencing system includes initiating a counter tosaid local change.
 18. The computer-implemented method of claim 16wherein said assessing includes predicting a local change in said localenvironment producing a predicted local change and wherein said stepsa)-c) are included in a control system for a building, wherein saidbuilding includes an automated climate control responsive to saidcontrol system, and wherein said predicted local change is processedinto control signals for said control system which operates saidautomated climate control to reduce a magnitude of said actual localchange or a rate of change of said actual local change.
 19. Thecomputer-implemented method of claim 16 wherein said assessing includespredicting a local change in said local environment producing apredicted local change and wherein said steps a)-c) are included in adistributed meteorological system having a plurality of data gatheringlocations distributed over a geographical region having a plurality ofmicroclimates greater in number than a number of said data gatheringlocations, wherein each said data gathering location performs stepsa)-c), and wherein a statistical processing of said predicted localchanges for one or more of said microclimates.
 20. Thecomputer-implemented method of claim 16 wherein said assessing includespredicting a local change in said local environment producing apredicted local change and wherein said steps a)-c) produce a predictedlocal change for a first building connected to a power grid operated bya utility company, wherein said predicted local change for said firstbuilding is accompanied by a change in an energy demand by said firstbuilding through said power grid wherein said change in said energydemand determines whether said utility changes an energy production forsaid power grid, and wherein said predicted local change is provided tosaid utility allowing said utility to modify said energy productionconsistent with said actual local change.